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Mutual Information as a Tool for Identifying Phase Transitions in Dynamical 

Complex Systems with Limited Data 

R. T. Wicks, S. C. Chapman, and R. 0. DendjQ 

Centre for Fusion, Space and Astrophysics, University of Warwick, Coventry, CV4 7AL, UK. 

(Dated: February 2, 2008) 

We use a well known model (T. Vicsek et al. Phys Rev Lett 15, 1226 (1995)) for flocking 
to test mutual information as a tool for detecting order-disorder transitions, in particular when 
observations of the system are limited. We show that mutual information is a sensitive indicator of 
the phase transition location, in terms of the natural dimensionless parameters of the system which 
we have identified. When only a few particles are tracked, and when only a subset of the positional 
and velocity components are available, mutual information provides a better measure of the phase 
transition location than the susceptibility of the data. 

PACS numbers: 05.70.Fh, 89.75.-k, 05.45.Tp, 89.70.+C 



I. INTRODUCTION 

Order-disorder transitions are often found in complex 
systems. They have been identified in physical systems 
such as Bose-Einstein condensates and ferromagnets and 
in biological, chemical and financial systems. Phase tran- 
sitions are found, for example, in the behaviour of bac- 
teria [lj, locusts H, voting games and utilisation of re- 
source in markets [3| . These systems have in common the 
property that there is competition between fluctuations 
driving the system towards disorder, and inter-element 
interactions driving the system towards order. Insight 
into such systems can be gained using simple models. 
Although the dynamics of individual elements are diffi- 
cult to predict, one can identify macroscopic parameters 
that characterise the behaviour of the system. These can 
be approached through dimensional analysis, e.g. Buck- 
ingham's II theorem [4|. 

A generic challenge in real world measurements of 
physical, chemical, biological or economic systems is that 
they yield datasets that are, in essence, sparse. Single el- 
ements such as tracer particles in turbulent flow, tagged 
birds or dolphins in a group or a constituent of a finan- 
cial index may, or may not, adequately sample the full 
underlying system behaviour. In consequence, the be- 
haviour of a finite number of individual elements may, or 
may not, provide a proxy for the behaviour of the entire 
system. If the system behaviour is known to exhibit a 
phase transition, the question arises how this can best 
be captured from analysis of the dynamics of individual 
elements. Previously, for example, both mutual infor- 
mation (MI) H @] [3] [| 

and susceptibility have been 
shown to be sensitive to the phase transition in the Ising 
spin model of ferromagnetism Q. MI can also extract 
correlation, or dependence, between causally linked but 
spatiotemporally separated observed parameters: for ex- 
ample, between in-situ plasma measurements in the solar 



wind, and the ionospheric res pon se detected by ground 
based measurements on Earth 11 Oil : and within the brains 
of Alzheimer's disease patients fill ]. 

Here we compare the use of MI and susceptibility to 
quantify the location of the phase transition in the dimen- 
sionless parameter space of the Vicsek et al. model [l§] • 
There are numerous statistical methods for analysing sys- 
tems with many degrees of freedom, dating back to work 
by Helmholtz and Boltzmann in the 19th century, see for 
example [l3[ [l4[ and references therein. The microscopic 
behaviour of the Vicsek system, described below, does 
not conserve energy or momentum and this precludes a 
microscopic understanding of energy, momentum or any 
related quantities in the system. The driving character- 
istic of the system is entropy: this is augmented by the 
random forcing of the particles, and removed by their 
mutual interactions, which create correlation. With this 
in mind, we examine mutual information as a natural 
choice for capturing the entropy flow in the system, and 
characterising the system state with respect to its order 
parameters. 

We find that when full knowledge of the system is 
available, that is when all the particles are tracked, the 
susceptibility is an accurate method for estimating the 
position of the phase transition in the Vicsek model. 
However, if the data is limited to a sample of just a 
few particles out of a large number, or a subset of the 
complete data, this method is less accurate. 

We show that the mutual information of only a few 
particles, or of limited data from the whole system, can 
successfully locate the phase transition in dimensionless 
parameter space. For example we find that the MI of a 
timeseries of components of particle position or velocity is 
sufficient. We thus show that MI can provide a practical 
method to detect order-disorder transitions when only a 
few particles, or elements, of the system are observed. 



II. THE VICSEK MODEL 
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In 1995 Vicsek et al. [T3| introduced the self propelled 
particle model in which particles have a constant speed 



2 




FIG. 1: Multiple particles interact if within a radius 7? of 
each other. Each of the Nr particles within R (here Nr — 4) 
contributes its angle of propagation to the average (6 n R ), 
which is assigned to the particle at the centre of R. 



\v\ = vq and a varying direction of motion 9. In the 
discrete time interval St = f n +i — t n an isolated particle 
increments its vector position x n — > x n+1 by moving with 
constant speed vq in a direction 9 n which is in turn incre- 
mented at each timestep. In the model, particles interact 
when they are within distance R of each other, such that 
the direction of their motion tends to become oriented 
with that of their neighbours. This interaction is imple- 
mented at each step, as shown in Fig[TJ by replacing the 
particle's direction of motion 9 n by the average of those 
particles Nr within distance R, so that 9 n+ i = (9^ a ) 
with a random angle S9 n also added. The random fluctu- 
ation 59 n is an independent identically distributed angle 
in the range —rj < 59 n < 77, where r\ characterises the 
strength of the noise for the system. Thus for the i 
particle in the system, after n timesteps: 

dn+l = & + &8t (1) 

oi+i = K R ) + soi (2) 

Hn = v (cos 9 l n x + sm6£|) (3) 

Here, direction is defined by the angle from the x axis 
(x), and 77 is such that 77 = rj5t, that is normalised to the 
time step St. 

There are two limiting cases for the system dynam- 
ics: disorder, where each particle executes a random 
walk; and order, where all particles move together with 
the same velocity. Figure shows snapshots of the sys- 
tem dynamics for r\ = 0, 77 = 27r/5 ~ 1 and 7/ = 47r/5 > 1. 
We see that 77 <c 1 is highly ordered and 77 3> 1 is highly 
disordered, and around 77 ~ 1 there is a phase transition 
As with other critical systems it is possible to de- 
fine an order parameter (f> and a susceptibility \ [3 [3 
[l7| [l8[ . For the Vicsek model, the magnitude of the av- 
erage velocity of all the particles in the system provides 
a macroscopic order parameter and the variance of this 




FIG. 2: The effect of increasing noise on a typical Vicsek 
system from ordered dynamics (top: 77 = 0) to disordered 
dynamics (bottom: 77 = 4tt/5), and in the vicinity of the 
phase transition (middle: 77 = 2n/5). Particle velocity vectors 
are plotted as arrows at the position of each particle in the 
x-y plane. The system has parameters TV = 3000, \v\ = 0.15, 
R = 0.5. This corresponds to II2 = 0.3, H3 = 0.94 and 
IIi = 77, see equations 16181 
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FIG. 3: (Colour online) An example of a typical Vicsek sys- 
tem. The order parameter <j> (line) is maximum for zero noise 
and falls to a constant small value at high noise. The suscep- 
tibility x (crosses) peaks at the critical point rj c » 1.33 for the 
system. The system parameters are: II2 = 0.3 and II3 = 0.94, 
with N = 3000. 



speed is the susceptibility: 



Nv 



N 



Ea 4 



(4) 
(5) 



Here N denotes the total number of particles in an im- 
plementation of the model of Vicsek et at. 

We plot 4> and x as a function of 77 in Fig [3] In the 
thermodynamic limit (N — > 00, 1 — ► 00) where I is the 
system size, the susceptibility would tend to infinity at 
the critical noise r) c , where the phase transition occurs. In 
a finite sized realisation of system, the susceptibility has 
a sharp but finite maximum at the critical noise at which 
the phase transition occurs. Finite size effects make the 
peak location uncertain, but it is still possible to obtain 
an estimate of the critical noise r\ c . 

The system can be analysed using Buckingham's II 
theorem [4[, and three independent dimcnsionless quan- 
tities can be formed that characterise its behaviour. The 
first of these (III) is the amplitude of the normalised noise 
77, the second (II2) is the ratio of the distance travelled 
in one timestep voSt to the interaction radius R, and the 
third (II3) is the average number of particles contained 
within a circle of one interaction radius R: 



III = V = TjSt 
H-2 = v St/R 
n 3 = irR 2 p 



(6) 
(7) 
(8) 



These three parameters determine the behaviour of the 
system in the thermodynamic limit (N — > 00, I — > 00, 
R and p finite) where p denotes the number density of 
particles over the whole system. 




FIG. 4: (Colour online) Phase transition diagram contours 
for the Vicsek model around II3 = 1. Top panel: the effect 
of changing II3, from II3 = 0.2 (dark blue contours, left hand 
side) to II3 = 2.0 (dark red contours, right hand side) in 
steps of 0.2, on the position of the phase transition in the 
IIi,Il2 plane. Bottom panel: the effect of changing 1T2, from 
II2 = 0.05 (dark blue, left hand side) to II2 = 0.5 (dark red, 
right hand side) in steps of 0.05, on the position of the phase 
transition in the IlijIIa plane. 



The system size / affects the number of interactions 
that occur. If / is finite and the system is periodic as here, 
the finite system size increases the chance of two ran- 
domly chosen particles interacting, compared to the limit 
of infinite I. The system only approaches the thermody- 
namic limit when the finite interaction radius R -C I- 
Conversely, for example, if the interaction radius is half 
the diagonal size of the system, then all the particles 
interact with each other at any given moment. This im- 
plies a fourth parameter reflecting the finite size of any 
computer based realisation of this model: 



n 4 = R/l 



(9) 



In the thermodynamic limit we have N — > 00, 1 — > oo, 
whilst R and p = N/l 2 are finite, so that II4 — > and 
ITi_3 are finite and specify the system. 



III. SYSTEM PHASE SPACE 

For given values of LI2 and II3 , we run simulations of 
the Vicsek system for a range of values of III to deter- 
mine the value III = T\\ at which the susceptibility \ 
peaks and thus the phase transition occurs. By repeat- 
ing this operation for a set of parameter values of IT2 
and LI3, we obtain the full set of coordinates at which 
the phase transition is located for a region of the phase 
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space around II3 = 1. We show this graphically in Fig [4] 
where we plot contours of II|(IIi, n 2 ) in the upper panel, 
and 112(111,113), in the lower panel. These plots confirm 
that there is a smooth, well defined surface of ITj, II2, IT3; 
they can be used to inform the choice of ITi , 1T2 and II3 
for the next section. 

In relation to recent work by Nagy et al. [l9j . we 
see from Fig. [4] that the speed ^0 of the particles, which 
is contained within IT2, has a characteristic effect above 
vq w 0.3. The bottom panel of Fig2]shows that, for con- 
stant II3 , the phase transition becomes hard to detect and 
the contours start to break up as II2 is increased above 
approximately 0.3. This is a complementary demonstra- 
tion of the statement made in [19| that the phase tran- 
sition becomes first order in the high velocity (vq > 0.3) 
regime. 



IV. MUTUAL INFORMATION 

Mutual information (MI) quantifies the information 
content shared by two signals A and B. For discrete 
signals we can write the MI as 



I(A,B) 



P(a„bj)\og 



(10) 



Here the signal A has been partitioned into an al- 
phabet (a library of possible values the signal can take) 
A = {ai, . . . , aj, . . . a m } where a\ and a m are the extrema 
of A found in all data considered. The discretized signal 
takes value ai with probability P{a{) and similarly for 
bi we have P(bi), while P(a,-, &j) is the joint probability 
of Oj and bj occurring together. The chosen base of the 
logarithm defines the units in which the mutual informa- 
tion is measured. Normally base two is used, so that the 
mutual information is measured in bits. If we define the 
entropy of a signal as 

m 

H(A) = -^P(a i )log 2 (P(a i )) (11) 

i 

then MI can be written as a combination of entropies [f| 



I(A,B) = H(A) + H(B) - H{A, B) 



(12) 



The calculation of the entropies needed to form the 
MI is not trivial, as there is some freedom in the method 
of discretization of the signals and in the method used 
to estimate the probabilities P(eti), P(bj) and P(ai,bj). 
There are many different methods currently used, sum- 
marised and compared by Cellucci et al. [6( and Kraskov 
et al. 0). 

MI has been used in the analysis of the two dimen- 
sional Ising model by Matsuda et al. [£►]. Importantly 
the critical temperature for the Ising model is picked out 



precisely by the peak in the mutual information of the 
whole system. This peak survives the coarse graining 
of the system very well, which raises the possibility of 
mutual information being useful in the study of other 
complex systems. 



V. IDENTIFYING THE PHASE TRANSITION 

A. Full System Mutual Information 

In the 2D Vicsek system there are three variables for 
each of the N particles: their positions (x%y l ) and the 
orientation of their velocities 9 l , giving three signals X, 
Y and 8 each containing N measurements at every time 
step. The simplest discretization of these signals x l , y*, 
9 % is to cover the range of the signals with equally spaced 
bins, so for position coordinate X we have m bins Xi 
with width 5X. Then if n particles are in the range 
(Xi — Xi + SX) we have probabilities: 



P(X. 



n{X t 



NSX 



(13) 
(14) 



The single and joint probabilities P(Yj), P(©fc), 
P(Xi, 0fc) and P(Yj, 0&) are calculated in a similar man- 
ner. 

The key factor governing the accuracy with which 
MI is measured is to optimise the size of the bins used 
in the above procedure. If the bins are too large then 
resolution is lost, and the exact level of small scale struc- 
ture and clustering cannot be identified. If the bins are 
too small then at high noise the probability of finding a 
particle at a given point does not become smoothed over 
the whole system because individual particles can be re- 
solved, giving P(xi,yj) 7^ P(xi)P(yj) even though the 
system is in a well mixed random state. 

There is no ideal bin structure yet determined for 
this method of MI calculation Q Q • The Vicsek model 
has two natural length scales, R the interaction radius 
and I the box size, so that a good length scale to choose 
for discretization, when a snapshot of the whole system 
is being used, is the interaction radius R. Thus all our 
mutual information calculations made on the whole sys- 
tem use a bin size of 2R, the diameter of the circle of 
interaction; the bins are therefore squares of size 4i? in 
the (x, y) plane. When 9 is discretized the same number 
of bins are used as for x or y because there is no natural 
size for bins in 9. 

Given full knowledge of Xi,yi, 9i for all N particles 
in the system over a large number of timesteps, several 
different calculations of mutual information can be made. 
We find that the most accurate form of mutual informa- 
tion for the whole system is that calculated between x or 
y position and 9. Thus we perform the following calcu- 
lation at each time step n once the system has reached a 
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FIG. 5: (Colour online) The mutual information, / (circles) 
defined by Eq. (|I7[) peaks at approximately the same point as 
the susceptibility, \ (crosses) denned by Eq. ((5} the critical 
noise r/ c ~ f.33 is marked. The system parameters are 1I2 = 
0.15 and II3 = 0.98, with N = 3000 particles. Error bars 
on the susceptibility are largest around rj c , unlike those on 
mutual information. 



stable state: 

i(Y,e) : 



I{X,e)+I(Y,Q) 



(15) 
(16) 
(17) 



and average over all timesteps for which MI is measured. 

We compare the MI as calculated using the above 
method and the susceptibility as a function of normalised 
noise 77 in Fig [5l At large 77 the MI falls to zero as X, Y 
and 9 tend to uncorrelated noise (see also Q). We would 
also expect the MI to fall to zero at sufficiently low 77 as 
the system becomes ordered and this behaviour is also 
seen within the errors. The errors on our measurements 
of MI are calculated from the standard deviation of mea- 
surements of MI calculated over 50 simulations at each 
noise rj. The error on the susceptibility is calculated in 
the same manner. 

The error bars become larger at low 77 because the 
mutual information includes the signatures of spatial 
clustering as well as velocity clustering in the measure- 
ment. Thus at low 77, when extended clusters form, the 
mutual information will give a higher value for the more 
spatially extended axis of the cluster and a lower value for 
the less extended axis of the cluster. This implies that the 
shape and orientation of the (usually single) large cluster 
formed at low noise influences the mutual information. 
Different measurements of MI thus arise for each imple- 



mentation of the model, giving rise to the error seen at 
low 77. This could be corrected by using other approaches 
to computing MI, for example recurrence plotting Q [l(| , 
or a different distribution of bins; these are more compu- 
tationally intensive, however. 

When estimated as the standard deviation over 50 
repeated runs of the simulation, the error is found to 
be considerably larger, as a fraction of the overall mea- 
surement, for the susceptibility than for the MI. This is 
because the susceptibility is simply an average fluctua- 
tion over all the velocity vectors of the system; whereas 
the MI also directly reflects the level of spatial 'dumpi- 
ness' (that is, spatial correlation) of the particles. The 
detailed spatial distribution varies from one simulation 
to the next, but at fixed IIi_4 the degree of 'dumpiness' 
does not. Mutual information is able to quantify cluster- 
ing (correlation in space as well as velocity) in a simple 
dynamical complex system, in a manner that identifies 
the order-disorder phase transition. 



B. Mutual Information from Limited Data 

Observations of many real world systems typically pro- 
vide only a subset of the full system information, which 
here comprises the positions and velocities of all N inter- 
acting particles. We now consider results from the Vicsek 
model using only very limited amounts of data. The mu- 
tual information and susceptibility are now calculated on 
a t = 5000 step timeseries of positional and velocity data 
for n = 10 particles out of the N = 3000 simulated. To 
optimise both methods, the data for each particle time- 
series is cut into S sections, labelled s = 1, . . . , S of length 
N s = t/S steps. This gives us nS pseudo-systems, re- 
lying on the assumption that one particle over N s steps 
is equivalent to N s particles at one step. This is a rea- 
sonable assumption to make for the Vicsek model as it is 
ergodic while 77 remains constant. 

To calculate the susceptibility, we need to estimate 
the variance of the average velocity of each of these nS 
pseudo-systems. We therefore cut each section s into S' 
further subsections s', calculate the average velocity 
of these subsections and find their variance, giving \\ 
the pseudo-system susceptibility. This is done for each 
pseudo-system individually to give %s) and averaged over 
all nS pseudo-systems to give %, the average variance of 
the average velocity for all pseudo-systems: 



SS' 

1 



r/SS' 

E 2* 



k=l 
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xl = ^«^>-«v> 2 ) 



S n 



x 



(18) 

(19) 
(20) 



s—1 i=l 

The result is shown in Fig [(J] where we also plot the 
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FIG. 6: (Colour online) The mutual information / (circles) 
calculated using timeseries from only ten particles for 5000 
time steps, with S = 10, compared to the average suscepti- 
bility x (crosses) for the same data using S" = 10 subsections 
to calculate x an d with the critical noise rj c ~ 1.33 marked. 
System parameters are 1I2 = 0.3 and II3 = 0.94, with N 
N = 3000 particles. 



mutual information I(X, 0) from Eq. (|15[) . but now as 
nS timeseries; the parameters used are n = 10, S = 10, 
S' = 10. The error bars are calculated as the standard 
deviation of the 100 measurements made using the dif- 
ferent pseudo-systems of length t/S — 500 timesteps. 
These values for n, S and S' are chosen so as to limit the 
data in a realistic way. n = 10 is a suitably small subset 
of the N = 3000 particles. S = 10 cuts the data into 
segments sufficiently long (500 timesteps) to be treated 
independently. S' = 10 is chosen so that each section s' 
is still long enough (50 timesteps) to make as good an 
estimate of the average velocities (f> l s , as possible, but al- 
lows enough of these measurements to be made to reduce 
the error on the measurement of \l- 

The system is identical to that shown in Fig [5] and 
the phase transition is at the same noise, r\ c ~ 1.33. Near 
their respective peaks, the error in the mutual informa- 
tion remains smaller than that in the susceptibility and 
so MI better identifies the peak. The peak in the sus- 
ceptibility no longer coincides with r\ c and is shifted to 
the higher noise side of the phase transition. This occurs 
because the susceptibility is now measured on too small 
a sample of data: only 50 angles 9\ are averaged to find 
each subsection velocity , . Such a small ensemble av- 
erage results in a large deviation in the average velocity 
from the expected value. 

For comparison with a linear measure, we calculate 
the cross correlation for our ten trajectories. We choose 
one of the particles at random and compute its cross 
correlation with each of the remaining nine. The aver- 
age of these is plotted in Fig [7] The average cross cor- 
relation between angles 8 1 and 9 k ,2 < k < 10 in the 
top panel shows strong correlation at low noise, as ex- 
pected. This cross correlation declines as noise increases, 
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FIG. 7: The cross correlation between a randomly chosen 
particle and nine others, calculated using a timeseries with 
5000 steps. The top panel shows the average cross correlation 
between 8 1 and 6 k ,2 < k < 10. The bottom panel shows 
the average cross correlation between x 1 and x k , 2 < k < 10. 



System parameters arc II2 
3000 particles. 



0.3 and II3 



0.94, with N 



but not smoothly, because the correlation depends on 
the exact dynamics of the particles considered. Angular 
cross correlation reaches zero around the phase transi- 
tion, but does not provide an accurate location for the 
critical noise. In the bottom panel of Fig [7] the cross 
correlation between x 1 and x k ,2 < k < 10 provides no 
reliable indication of the position of the phase transition. 
The cross correlation does become more variable on the 
higher noise side of the graph but this effect cannot be 
used to accurately find the critical noise r\ c . 

The value of using mutual information can be seen 
when the available data is restricted still further. Let 
us consider signals derived from one component of the 
particle trajectory only, equivalent to a line of sight mea- 
surement. We then have one of the position coordinates 
x\, and the instantaneous x component of the velocity, 
Ax], = vqcos(6\). The susceptibility is calculated as in 
Eqs (fl~5]) - (j!i0|) , but using the average one dimensional 
velocities Axi: 
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FIG. 8: The mutual information I(X, AX) (circles) calcu- 
lated using a timeseries from only ten particles for 5000 
time steps, with S = 10 and the critical noise r\ c m 1.33 
marked. System parameters are II2 = 0.3 and II3 = 0.94, 
with N = 3000 particles. 
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FIG. 9: The susceptibility \ (crosses) calculated using a time- 
series of one dimensional data {X, AX} from only ten parti- 
cles for 5000 time steps, with S = 10 and the critical noise 
r\ c ~ 1.33 marked. System parameters are 1T2 = 0.3 and 
n 3 = 0.94, with N = 3000 particles. 



The mutual information is calculated for each section of 
the x only (and later y only) components of the timeseries 
for each particle using a suitable binning: 



I(X,AX) = Y,P(X i ,AX j )log : 



f P(X t ,AX,) 
{p(X i )P(AX j 



(24) 



I(Y,AY) =E^,Ar,)log 2 ( pffi^ ) (*) 



1,3 



Figure [5] shows the mutual information calculated 
from the data in this manner with S = 10. The peak in 
the mutual information is at approximately the correct 
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FIG. 10: (Colour online) The mutual information I(X, AX) 
(circles) and I(Y, AY) (squares) calculated using a timeseries 
from only ten particles for 5000 steps, with 5 = 1 and the 
critical noise rj c » 1.33 marked. System parameters are II2 = 
0.3 and II3 = 0.94, with N = 3000 particles. 



value of 77 (rjc w 1.33). Figure [9] shows for comparison the 
susceptibility calculated over the X data as in equations 
(pH]) - (j2"3")) . We see that although there is a peak, it no 
longer identifies 77 — > r\ c accurately. The peak is broader 
and has larger error bars than in Fig giving a large 
uncertainty in identifying r\ c . 

The peak in Fig [8] is shifted to the low noise side of 
the phase transition and shows some scatter. This can be 
understood by looking at the same data using a different 
value of the interval S. In Fig [TO] we show the same data 
analysed using S = 1, that is we consider one timeseries 
of length 5000 time steps for each of ten particles and ob- 
tain MI averaged over these ten. We plot I(X, AX) (cir- 
cles) and I(Y, AY) (squares). The measurements overlap 
within errors on the high noise side of the phase transi- 
tion but separate into two distinct branches, containing 
both I(X, AX) and I(Y, AY), on the low noise side. 

One potential source of this behaviour is that, as the 
system becomes ordered at 77 < r) c , the particles clump 
together. This implies that the particles together take on 
a preferred direction of motion; in addition, a clump may 
be elongated in a particular spatial direction. The effec- 
tiveness of MI will then depend on whether our single 
component (line of sight) data is aligned along, or per- 
pendicular to, these key directions. Mutual information 
measured in terms of coordinates aligned with the pre- 
ferred direction of motion is increased by the dispersion 
of particle positions and velocities, whereas MI measured 
in term of perpendicular coordinates is decreased because 
the positional and velocity dispersion are smaller in this 
direction. Anomalously high MI measurements result 
from making measurements along the preferred direction 
of motion; large relative velocities lead to anomalously 
high peaks on the low noise side of the phase transition, 
making it appear to be shifted towards 77 = 0. 
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FIG. 11: The minimum results from mutual information mea- 
surements I(X, AX) and I(Y, AY) calculated using a time- 
series from only ten particles for 5000 steps, with S = 1 and 
the critical noise n c ~ 1.33 marked. System parameters are 
n 2 = 0.3 and II3 = 0.94, with N = 3000 particles. 

Finally in Fig [11] we plot the minimum of I(X, AX) 
and I(Y, AY) for each value of rj from FigflOland see that 
a clear peak emerges at rj — rj c , where the error bars are 
smallest. This outcome obviates the difficulty that arises 
if we only allow knowledge of {X, AX} for example, when 
it would be necessary to exclude high measurements of 
Ml at low noise, as discussed above. 

VI. CONCLUSIONS 

The Vicsek model [l2j is used here to test the potential 
of measurements of order and clustering that exploit mu- 
tual information in dynamic complex systems. We find 
that when complete knowledge of the system is available, 
the mutual information has a smaller error than the sus- 
ceptibility (Fig [5]). Using Buckingham's II theorem the 
set of dimensionless parameters that capture the phase 
space of the Vicsek model have been presented as a com- 



plete set for the first time. 

When data is limited to observations of only ten par- 
ticles out of 3000, the error in the mutual information re- 
mains comparatively small, and the mutual information 
thus provides a better measurement than susceptibility 
of the position of the order-disorder phase transition (Fig 
When data is limited still further, such that only one 
line of sight component of the particle motion is avail- 
able, the mutual information measurement remains sen- 
sitive enough to identify the critical noise of the phase 
transition, while the susceptibility does not (Figs 151 - [TTj) . 

In this case the mutual information also provides 
an indication of the preferred axial direction of clumped 
particle motion at low noise. Anomalously high mutual 
information estimates in this ordered phase indicate that 
the particles measured are mostly moving along the di- 
mension being measured; low estimates indicate that the 
particles are moving perpendicular. This is remarkable 
given that susceptibility does not contain this informa- 
tion, and that the MI is a probabilistic measurement. 

Real world data are often in the form of the final 
data studied here; a limited sample from a much larger 
set, measured in fewer dimensions than those of the 
original system: for example, line of sight measurements 
of wind speed measured by an anemometer at a weather 
station, or satellite measurements of the solar wind. 
It has been shown here that mutual information can 
provide an effective measure of the onset of order, and 
may provide a viable technique for real world data with 
its inherent constraints. 
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